A Tale of Two DRAGGNs: A Hybrid Approach for Interpreting Action-Oriented and Goal-Oriented Instructions

摘要

Robots operating alongside humans in diverse, stochastic environments must beable to accurately interpret natural language commands. These instructionsoften fall into one of two categories: those that specify a goal condition ortarget state, and those that specify explicit actions, or how to perform agiven task. Recent approaches have used reward functions as a semanticrepresentation of goal-based commands, which allows for the use of astate-of-the-art planner to find a policy for the given task. However, thesereward functions cannot be directly used to represent action-oriented commands.We introduce a new hybrid approach, the Deep Recurrent Action-Goal GroundingNetwork (DRAGGN), for task grounding and execution that handles naturallanguage from either category as input, and generalizes to unseen environments.Our robot-simulation results demonstrate that a system successfullyinterpreting both goal-oriented and action-oriented task specifications bringsus closer to robust natural language understanding for human-robot interaction.